Three Heads are Better than One

Machine translation (MT) systems do not currently achieve optimal quality translation on free text, whatever translation method they employ. Our hypothesis is that the quality of MT will improve if an MT environment uses output from a variety of MT systems working on the same text. In the latest version of the Pangloss MT project, we collect the results of three translation engines typically, subsentential chunks in a chart data structure. Since the individual MT systems operate completely independently, their results may be incomplete, conflicting, or redundant. We use simple scoring heuristics to estimate the quality of each chunk, and find the highest-score sequence of chunks (the "best cover"). This paper describes in detail the combining method, presenting the algorithm and illustrations of its progress on one of many actual translations it has produced. It uses dynamic programming to efficiently compare weighted averages of sets of adjacent scored component translations. The current system operates primarily in a human-aided MT mode. The translation delivery system and its associated post-editing aide are briefly described, as is an initial evaluation of the usefulness of this method. Individual M T engines will be reported separately and are not, therefore, described in detail here. 95 1 I N T R O D U C T I O N Current MT systems, whatever translation method they employ, do not reach an optimal output on free text. In part, this is due to the inherent problems of a particular method for instance, the inability of statistics-based MT to take into account longdistance dependencies, the difficulty in achieving extremely broad coverage in knowledge-based MT systems, or the reliance of most transfer-oriented MT systems on similarities in syntactic structures of the source and the target languages. Our hypothesis is that if an MT environment can use the best results from a variety of MT systems working simultaneously on the same text, the overall quality will improve. Using this novel approach to MT in the latest version of the Pangloss MT project, we submit an input text to a bat tery of machine translation systems (engines), collect their (possibly, incomplete) results in a joint chart data structure and select the overall best translation using a set of simple heuristics. 2 I N T E G R A T I N G MULTI-ENGINE O U T P U T In our experiment we used three MT engines: * a knowledge-based MT (KBMT) system, the mainline Pangloss engine (Frederking et al., 1993b); • an example-based MT (EBMT) system (see (Nirenburg et al., 1993; Nirenburg et al., 1994b); the original idea is due to Nagao (Nagao, 1984)); and • a lexical transfer system, fortified with morphological analysis and synthesis modules and relying on a number of databases a machine-readable dictionary (the Collins Spanish/English), the lexicons used by the KBMT modules, a large set of user-generated bilingual glossaries as well as a gazetteer and a list of proper and organization names. The outputs from these engines (target language words and phrases) are recorded in a chart whose positions correspond to words in the source language input. As a result of the operation of each of the MT engines, new edges are added to the chart, each labeled with the translation of a region of the input string and indexed by this region's beginning and end positions. We will refer to all of these edges as components (as in "components of the translation") for the remainder of this article. The KBMT and EBMT engines also carry a quality score for each output element. The KBMT scores are produced based on whether any questionable heuristics were used in the source analysis or target generation. The EBMT scores are produced using a technique based on human judgements, as described in (Nirenburg et al., 1994a), submitted.